A content delivery network (CDN) is a distributed platform that delivers the content and services of different providers on behalf of those providers. The CDN distributed platform retrieves the content and services from the providers' servers and fans out the retrieved content and services to different clients or users across a digital network, such as the Internet, while also caching the content and services. The caching allows the distributed platform to serve the content and services in response to subsequent client requests without the need or delay associated with re-retrieving the content and services from the providers' servers.
The CDN distributed platform is formed by one or more points-of-presence (PoP). The PoPs are located in different geographic regions. Each PoP is tasked with delivering the content and services to the one or more regions that receive optimal content delivery performance from the PoP. The optimal performance can be based on geographic or network node proximity to the one or more regions or other factors that affect the content delivery performance to those one or more regions. Each PoP includes one or more request directors and two or more back-end servers.
The request directors include load balancing devices or other devices that distribute requests amongst the back-end servers. The request directors receive requests issued by different clients over a digital network. The requests can be embodied as HyperText Transfer Protocol (HTTP) GET requests as one example. The issued requests are distributed across the back-end servers and the back-end servers respond to the requests by serving the requested content or services.
To maximize performance and efficiency, the request directors attempt to provide a consistent distribution of requests, whereby requests for the same content or service are distributed to the same server. Such consistent request distribution optimizes performance and efficiency of the back-end servers by allowing different back-end servers to specialize in serving a subset of the content or services served by the PoP rather than generally serving all such content or services at different times. More specifically, the consistent request distribution increases the cache hit ratios of the servers and reduces cache rewriting, thereby allowing the servers to respond to a greater number of requests directly from cache without having to retrieve the requested content or service from an origin site of the originating content or service provider.
The consistent request distribution is based on the Uniform Resource Locator (URL) or Uniform Resource Identifier (URI) specified in the requests. For instance, the request directors may hash the request URL in order to identify which of the back-end servers is tasked with serving the requested content or service identified by the URL. The Cache Array Routing Protocol (CARP) is one URL hash based request distribution scheme. The terms URL and URI are used interchangeably hereafter.
The request URL does not, however, consistently and accurately identify the content or service being requested in all instances. Different dynamic content, unique customized content, or other content and services can be requested using the same request URL. In such cases, the requested content or service is differentiated by other cache keys besides the request URL. The cache keys include one or more query string parameters appended to or otherwise included with the URL or header parameters that differentiate the content or service being requested.
Prior art request directors typically ignore the cache keys when performing the request routing decision and base the routing decision solely on the base URL. This is because not all cache keys are relevant to the routing decision, the ordering or placement of relevant cache keys can differ from request to request, and the number of cache keys for the same requested content or service can differ from request to request.
Ignoring cache keys in the routing decision is a simplistic means with which to preserve request distribution consistency. This consistency comes at the expense of reduced cache hit ratios and increased load on the server that receives the requests directed to the same URL but different content or services.
Hot file mechanisms within the PoP can dynamically allocate additional server resources to prevent the server designated to respond to requests directed to the same URL but different content from becoming overwhelmed and underperforming. However, the efficiency and performance of the additional server resources is also compromised as they too will experience reduced cache hit ratios because the request directors incorrectly triggered the hot file mechanisms thinking that the requests are directed to the same content because of the common URL even though the requests are actually directed to different content because of the included but ignored cache keys.
Accordingly, there is a need for a consistent request distribution scheme that consistently and accurately routes a request based on the request URL and one or more cache keys included with the request, wherein the cache keys include some but not necessarily all query string parameters or header parameters associated with the request. To this end, there is a need to track, differentiate, and base the request distribution on relevant cache keys that affect the routing decision while ignoring other irrelevant cache keys that do not affect the routing decision. There is further a need to maintain request distribution consistency when the requests provide a different ordering or positioning for the relevant cache keys or a different number of cache keys as part of the request.